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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BOARD OF PATENT APPEALS AND INTERFERENCES 



In re application of: Waltz et al. 
Serial No.: 09/863,513 
Filed: May 23, 2001 
I For: TEXT AND IMAGERY SPATIAL CORRELATOR 



o 



Group No.: 2121 
Examiner: Sunray Chang 



APPELLANTS' BRIEF UNDER 37 CFR §1.192 



Mail Stop Appeal Brief 
Commissioner for Patents 
PO Box 1450 

Alexandria, VA 22313-1450 
Dear Sir: 



o 

CL 

w 
o 



I. Real Party in Interest 

The real party and interest in this case is Veridian ERIM International, Inc., a Michigan 
corporation, by assignment. 

IL Related Appeals and Interferences 

There are no appeals or interferences which will directly affect or be directly affected by or have 
a bearing on the Board's decision in the pending appeal. 

III. Status of Claims 

The presentapplication was filed with 23 claims. Claims 13-14 were canceled by amendment in 
February 2005. Claims 1-12 and 15-23 are pending, rejected and under appeal. Claims 1 and 18 are 
independent. 



IV. Status of Amendments Filed Subsequent 
Final Rejection 



No after-final amendments have been made. 
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- V. Summary of Claimed Subject Matter 

CM 

? Independent claim 1 resides in a method of correlating text and imagery. The method includes 

o 

§ the steps of specifying a target concept, providing textual material and imagery, and training a text 
< 

| search detector to examine the textual material for text regions which relate to the target concept. Text 

y 

5 relating to target detection is recorded in a database A in the event of a match or other meaningful 

o 

£ association. The method further includes the step of training a discriminating feature detector to search 
I for locations within the imagery which relate to the target concept, and creating a location target 

X 

m detection record in a database B in the event of a match or other meaningful association. The records in 

q 

o" both databases are compared to declare an approximate correlation, if any, indicative of a common 

CO 
CO 

b target concept. (Specification, page 6, line 5 to page 8, line 19). 

of Independent claim 18 is directed to a system for a text and imagery spatial correlator. The 

Q 

S system comprises a document text parsing and interpretation engine which uses a context-based search 

z 

LU 

£ to generate topical information, and an imagery engine operative to associate the components of an 

o 

£ image with known spatial features and generate location information; and a matching subsystem 

5 

w operative to associate the topical information with the location information and present a result to a 

q 

^- user. (Specification, page 5, line 9 to page 6, line 3). 



^ VI. Grounds of Objection/Rejection To Be Reviewed On Appeal 

I 1. The rejection of claims 1-10 and 12-23 under 35 U.S.C. § 102(b) as being anticipated by 

1 U.S. Publication No. 2002/0156779 to Elliott. 

< 

si 2. The rejection of claim 11 under 35 U.S.C. §103(a) as being unpatentable over U.S. 

z 

% Publication No. 2002/0156779 to Elliott in view of U.S. Patent No. 6,741,743 to Stalcup. 



o 
cr 
o 



VII. Argument 

A. Claims 1-10 and 12-23, wherein claims 2-10 and 12-23 stand or fall with claim 1 . 

it Claim 1 was rejected under 35 U.S.C. § 102(b) over Elliott (U.S. Publication No. 

o 

2002/0156779). It is Appellant's position that the Examiner has misinterpreted both the claims on 
appeal and the cited reference. 
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S Elliot discloses an Internet search engine based upon a spatial indexing intelligent agent. The 

00 

~ agent indexes information against a database of spatial language which is used in combination with a 

cvi 

° modified search engine that conducts searches using spatially relevant criteria and spatial analysis 

o 

§ algorithms. Alpha-numeric values from a mathematical system are used for identifying spatial locations, 
< 

| and can be arbitrary, geocentric, virtual, and galactic ('779 application, Abstract). 

o 

* It is well-settled that anticipation may be established only when a single prior art reference 

o 

f discloses, expressly or under principles of inherency, each and every element of a claimed invention. 
§ RCA Corp. v. Applied Digital Data Systems , 730 F.2d 1440, 1444, 221 USPQ 385, 388 (Fed. Cir. 

X 

* 1984). A disclosure "that 'almost' meets that standard does not 'anticipate'." Connell v. Sears, Roebuck 

p 

o- Co., 722 F.2d 1542, 1548, 220 USPQ 193, 198 (Fed. Cir. 1983). Elliot does not teach or suggest 

CO 
CO 

b correlations involving imagery. Accordingly, there can be no anticipation. 

cr r On page 2 of the final Office Action, under a section entitled "Claim Interpretation," the 

Q 

S Examiner explains how "imagery" is to be treated for the purposes of examination. In particular, the 

z 

£ Examiner cites page 5, lines 11-16 of the instant specification, which reads as follows: 
o 

f 'The TISC [Text and Imagery Spatial Correlator] automatically detects and 

p correlates events ... or objects . . . that are observable in imagery and described in 



o 
a. 



CO 



text reports." (Emphasis added.) 



§ It should be clear from this passage - cited by the Examiner - that the invention of Appellant 

% does exactly what is stated in the specification; that is, images are compared to text. Nevertheless, for 

z 

8 some reason which escapes Appellant, the Examiner "interprets" imagery to mean "information of 
hi 

§ imagery." Why? Apart from the fact that Applicant does not see a need for such an interpretation, it 

LU 
Z 

cc 



appears to be incorrect. Imagery is imagery, and imagery is not in need of a more "sophisticated" 
% definition - especially "information of imagery," which has no clear meaning, 
g The Examiner argues that "the imagery is considered to be the 'coordinate information' in 

» Elliott." However, this appears to be entirely incorrect. Referring to the definition section of the Elliott 
q- application, paragraph [0019], "coordinate information" is defined as "alpha-numeric values from a 

DC 

it mathematical system for identifying spatial locations, and can be arbitrary, geometric, virtual, and 
galactic." "Alpha-numeric values" are not "imagery." Indeed, by "spatial," Elliott refers to coordinate 
information and identifier information, such as a city name, county, state, area code, zip code, and so 
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S forth. This is information about location, and not imagery. 

CO 

- In the Elliott reference, a comparison of textual material and imagery simply does not occur. 

° Indeed, the Examiner appears to have misinterpreted Elliot altogether. With respect to Appellant sstep 

5 of "providing textual material and imagery," the Examiner cites paragraphs 25 and 26 of Elliot, which 

< 

| read as follows: 
y 

^ "[0025] A second database, separate from the spatial lexicography database, contains 

§ documents indexed by a spatial indexing intelligent agent or spider. How the spider 

searches for documents will be discussed later. 



o [0026] Having both databases, a requester would provide search criteria which is 

o necessary to conduct the search. The search criteria comprises a reference location 

and a search radius about the reference location." 



CL 



w „ These passages do not teach the step of "providing textual material and imagery." The 

cc 

S Examiner claims that Elliott teaches "specifying a target concept," but this in and of itself appears to be 
i- 

§ untrue. Referencing paragraph [0026], it appears that a user of the Elliott system inputs "a reference 
> 

S location and a search radius about the reference location," and not a "target concept." The same holds 
§ true of the step of training a discriminating feature detective to search for locations within the imagery 
" relating to a target concept. Given that anticipation requires the disclosure of each and every element of 

| an invention as claimed, anticipation simply does not apply in this case. 

o 

o 

g B. Claim 11 

CO 

g Claim 1 1 stands rejected over Elliott in view of Stalcup ('743). With respect to the combination 

< 

uj of Elliott and Stalcup, the Examiner claims that it would have been obvious to combine these references 
i "for the purpose of efficient storage of the data and efficient indexing and retrieval of multi-media data 

CO 

g objects." Applicant has no idea from where this conclusion has been drawn, but in any event, there is 

cc 

no teaching or suggestion from the prior art as to the proposed combination, and only speculation on 
the part of the Examiner, prima facie obviousness has not been established. 
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- In conclusion, for the arguments of record and the reasons set forth above, all pending claims of 

OJ 

? the subject application continue to be in condition for allowance and Appellants seek the Board's 



concurrence at this time. 



< 

I 

O 



> 

o 

DC 



Respectfully submit! 



x Date: March 22, 2006 

o 
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John G. Posa 
Reg. No. 37,4 
Gifford, Krass, Groh, Sprinkle, 
Anderson & Citkowski, P.C. 
PO Box 7021 
Troy, MI 48007-7021 
(734)913-9300 
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APPENDIX A 



CLAIMS ON APPEAL 



z 1. A method of correlating text and imagery, comprising the steps of: 

o 

5 specifying a target concept; 

g providing textual material and imagery; 

training a text search detector to examine the textual material for text regions which relate to the 

o 

£ target concept, and creating a text target detection record in a database A in the event of a match or 

CO 

6 other meaningful association; 

CL 

§ training a discriminating feature detector to search for locations within the imagery which relate 

ID 

5 to the target concept, and creating a location target detection record in a database B in the event of a 
S match or other meaningful association; and 

cc 

ID 

z comparing the records in both databases to declare an approximate correlation, if any, indicative 



o of a common target concept. 

DC 
I- 

O 
CM 

2. The method of claim 1, wherein the target concept is an event or object. 



o 

CL 

w 

§ 3. The method of claim 1, wherein the discriminating features within the imagery include 

o 

06 infrared, multispectral or spatial features. 

o 

GO 

cc 

LD 
Q 

< 4. The method of claim 1 , wherein the step of training the text search detector includes the 

steps of: 

a) defining a search phrase; 



z 

cc 

CL 

</> 

I 
o 

5 b) testing the phrase against a validation set, and 

CO 
CO 



g c) repeating a) and b) until all relevant targets in the validation set are detected. 



Q 
CC 

o 



5. The method of claim 1, wherein the examination of the textual material includes 
searching the text regions for geographic location text associated with the target concept. 
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6. The method of claim 1, further including the step of generating a concept identifier code 
in both the text and image target detection records using a lookup table in the event of a match or other 
meaningful association. 



< 
o 

I 

o 
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7. The method of claim 6, wherein the searching of the text regions is accomplished by 
reference to a Gazeteer of place names and their corresponding lat-long locations. 

8. The method of claim 7, wherein the text target detection record contains: 
a text document ED number, 

an index to locate a paragraph or passage within the document, 
the target concept identifier code (CIC), and 
the latitude-longitude (LL) value. 

9. The method of claim 6, wherein the search for locations within the imagery includes 
extracting a lat-long location. 
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10. The method of claim 9, wherein the location target detection record contains: 
an image ID number, 

an index to locate the target within the image, 
the target concept identifier code (CIC), and 
the latitude-longitude (LL) value. 

1 1 . The method of claim 1 , wherein the target concept within the imagery is in the form of a 
pixel index. 

12. The method of claim 10, wherein: 

the searching of the test regions is accomplished by reference to a Gazeteer of place names and 
their corresponding lat-long locations; and 

the text target detection record contains: 



o 
o 
o 
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a text document ID number, 

an index to locate a paragraph or passage within the document, 
the target concept identifier code (CIC), and 
the latitude-longitude (LL) value. 



< 
a 
x 
o 



5 15. The method of claim 2, wherein the event may be characterized as an emergency, 

o 

f tragedy, disaster or crisis. 



m 16. The method of claim 2, wherein the object involves an environmental asset, structure, or 

q 

o mode of transportation. 



17. The method of claim 1 , wherein either or both of the steps associated with examining the 
S textual material or searching for locations within the imagery are carried out in a batch mode or as part 



lil 

£ of a recursive flow. 

o 

DC 

I- 

o 

CM 



18. A text and imagery spatial correlator, comprising: 

a document text parsing and interpretation engine which uses a context-based search to generate 
§ topical information; 

s an imagery engine operative to associate the components of an image with known spatial 

§ features and generate location information; and 

a matching subsystem operative to associate the topical information with the location 
information and present a result to a user. 



g 19. The text and imagery spatial correlator of claim 18, wherein text parsing and 

CO 

S interpretation engine includes a user-trainable agent to define the context of interest in a current search. 

o" 

cc 

I 20. The text and imagery spatial correlator of claim 18, wherein the topical information 

o 

concerns an event or an object. 
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S 21. The text and imagery spatial correlator of claim 20, wherein the event may be 

00 

~ characterized as an emergency, tragedy, disaster or crisis. 



o 
o 



22. The text and imagery spatial correlator of claim 20, wherein the object involves an 



< 

| environmental asset, structure or mode of transportation. 



o 

DC 



23. The text and imagery spatial correlator of claim 18, wherein the matching subsystem is 
operative to associate the topical information with the location information in a batch mode or as part of 



g 

m a recursive flow. 
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APPENDIX B 



EVIDENCE 
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None. 
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APPENDIX C 



RELATED PROCEEDINGS 



< 
CD 
X 

u 
o 



None. 
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